Reducing optical crosstalk and radial fall-off in imaging sensors

ABSTRACT

An imaging device and an image crosstalk analysis device are described that may be used to reduce optical crosstalk and radial fall-off in image data. The imaging device may include an image processing unit that applies corrections to the raw image data using newly developed correction algorithms and correction parameters that are generated for the imaging device by the image crosstalk analysis device based on an analysis of uncorrected image data generated by the imaging device. The correction parameters generated for a single imaging device may be loaded to imaging devices of the same make and model as part of the imaging device manufacturing process. Due to the flexible nature of the described processing techniques in which the control parameters are used, improved image quality may be achieved in every image generated by the respective manufactured cameras.

INCORPORATION BY REFERENCE

This application claims the benefit of U.S. Provisional Application No.61/074,890, “ONE DIGITAL IMAGE SIGNAL PROCESSING ALGORITHM TO REDUCEOPTICAL CROSSTALK AND RADIAL FALL-OFF FOR CMOS SENSORS,” filed by HansLi on Jun. 23, 2008, which is incorporated herein by reference in itsentirety.

BACKGROUND

Raw digital images generated by imaging devices, e.g., digital cameras,that use sensor arrays, such as a complementary metal oxidesemiconductor (CMOS) sensor arrays, may be degraded by one or more typesof distortion. The distortions can include, for example, spectralcrosstalk, electrical crosstalk, optical crosstalk, and sensor shading.

Spectral crosstalk may be caused by imperfect color filters that passsome amount of unwanted light of other colors. Spectral crosstalk maycause colors in a generated image to be unsaturated as compared to thecolor of the originally photographed object or scene. One approach forcorrecting spectral crosstalk is to apply a color correction matrix tothe raw image data that compensates for the spectral crosstalk.

Electrical crosstalk may result from photo-generated carriers within thesensor array of the imaging device moving to neighboring chargeaccumulation sites, i.e., neighboring sensors within the sensor array.Electrical crosstalk is a function of the underlying sensor images thatform the sensor array of the imaging device. As a result, the distortioncaused by electrical crosstalk is uniform across the entire image. Oneapproach for correcting electrical crosstalk is to apply a predeterminedcorrection factor to each sensor response value included in the rawimage data.

Optical crosstalk is the result of more complex origins. As described ingreater detail below, color filters included in the optical sensor unitof an imaging device are placed some distance from the pixel surface dueto metal and insulation layers. As a result, light coming at anglesother than orthogonal may pass through a color filter element for apixel, yet may pass diagonally to an adjacent pixel sensor rather thanto the pixel sensor associated with the color filter element throughwhich the light passed. Depending on the focal ratio of the lens, theportion of the light absorbed by neighboring pixel can varysignificantly. For this reason, optical sensor units may include anarray of micro-lenses layered over the array of color filter elements inorder to redirect the light in the direction of the intended pixelsensor. However, longer wavelengths, e.g., red light, are more difficultto bend than shorter wavelengths of light, e.g., blue light, so opticalcrosstalk may occur despite the use of such micro-lenses.

The level of the optical crosstalk depends on the wavelength of lightand the angle of incidence on the light on the individual pixels of thesensor array. Since the angle of incidence is related to pixel position,optical crosstalk is non-uniform for the whole image. Blue light is moreeasily bent by a micro lens and, therefore, is efficiently directed ontothe sensor at the bottom of the pixel, while red light is not easilybent by a micro lens and, therefore, may leak to adjacent pixels. Aslight travels from a center focused lens onto the sensors in a sensorarray, the angle of incidence of light on the respective pixelsincreases with the distance of the pixel from the center of the image,thereby increasing likelihood that light with longer wavelengths mayfall onto an adjacent pixel's sensor.

As a result of optical crosstalk, the center of an image may appearbrighter and redder and the surrounding portions of the image may appeardarker and bluer. Although optical crosstalk may be partially correctedfor using existing optical crosstalk correction, such techniques areoften inadequate.

Sensor shading is the result of pixels closer to the center of an imageshading pixels further from the center of the image. As described abovewith respect to optical crosstalk, color filters included in the opticalsensor unit of an imaging device are placed some distance from the pixelsensors due to metal and insulation layers. As a result, light coming atangles other than orthogonal may be shaded from reaching its intendedpixel by the metal and insulation layers above pixels closer to thecenter of the image. Further, lens shading is caused by the physicaldimensions of a multiple element lens. Rear elements are shaded byelements in front of them, which reduces the effective lens opening foroff-axis incident light. The result is a gradual decrease of the lightintensity towards the image periphery. Although sensor shading and lensshading are not wavelength dependent, as is optical crosstalk, suchsensor shading results in a similar form of radially increasingdistortion, i.e., radial fall-off, by decreasing the amount of lightreaching the pixels in the sensor array at locations further from thecenter of the image.

As a result of the combined effects of optical crosstalk and shading,pixels at the center of an image will experience minimal or no leakageand shading, but pixels further from the center of an image willexperience increasing levels of leakage, especially for red pixels, andincreasing levels of shading.

SUMMARY

Aspects of this disclosure describe an imaging device and an imagecrosstalk analysis device that may be used to reduce the effects ofoptical crosstalk and radial fall-off in image data.

The described imaging device may include an image processing unit thatmay correct for electrical crosstalk, spectral crosstalk, opticalcrosstalk and/or radial fall-off in both RGB and Bayer space formattedimage data. The image processing unit may apply corrections to the rawimage data using generalized correction algorithms that may be tailoredto the image correction needs of the imaging device with a plurality ofcorrection parameters. Such correction parameters may be generated andloaded to the imaging device based on an analysis performed using thedescribed image crosstalk analysis device.

The described image crosstalk analysis device may be used to analyzeimages generated by an imaging device. The image crosstalk analysisdevice may generate correction parameters that, when used in conjunctionwith the described processing techniques, may be used to correctelectrical crosstalk, spectral crosstalk, optical crosstalk and/orradial fall-off in future image data generated by the imaging device.The correction parameters may be uploaded to the imaging device and usedto correct raw image data in the imaging device itself, and/or may beretained within the image crosstalk analysis device for use incorrecting future images generated by the imaging device using the imagecrosstalk analysis device.

Depending on the uniformity with which the optical sensing units of therespective imaging devices are manufactured, once correction parametersare generated for a single representative imaging device, the samecorrection parameters may be loaded to imaging devices of the same makeand model as part of the imaging device manufacturing process. Due tothe flexible nature of the described processing techniques in which thecontrol parameters are used, improved image quality may be achieved inevery image generated by the respective manufactured imaging device.

In one example embodiment an imaging device is described that mayinclude, an optical sensing unit that may generate image data containinga plurality of color channels, a data storage unit that may store thegenerated image data and a plurality of image correction parameters, andan image processing unit that may process the stored image data colorchannels to remove optical crosstalk and radial fall-off distortionbased on the stored image correction parameters.

In a second example embodiment an image analysis device is describedthat may include, a sharpening matrix generation unit that may generatea sensor sharpening matrix that sharpens the effective sensor responseof a color channel in an image, and a radial fall-off curve generationunit that may generate a set of radial fall-off correction parametersthat may correct optical cross-talk and radial-falloff in the colorchannel in the image.

In a third example embodiment, a method of correcting optical crosstalkand radial fall-off distortion in image data is described that mayinclude, receiving a set of image correction parameters, generatingimage data containing a plurality of color channels, storing thegenerated image data and image correction parameters, and processing thestored image data color channels to remove optical crosstalk and radialfall-off distortion based on the stored image correction parameters.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments of an imaging device and an image crosstalk analysisdevice that may be used to reduce optical crosstalk and radial fall-offfor CMOS sensors will be described with reference to the followingdrawings, wherein like numerals designate like elements, and wherein:

FIG. 1 is a block diagram of an example imaging device that includes animage processing unit that may reduce optical crosstalk and radialfall-off in generated images;

FIG. 2 is a block diagram of an example optical sensing unit introducedin FIG. 1;

FIG. 3 is a block diagram of an example image processing unit introducedin FIG. 1;

FIG. 4 is a block diagram of an example Bayer space adaptation unitintroduced in FIG. 3;

FIG. 5 is a block diagram of an example image crosstalk analysis devicethat may be used to analyze images generate by an example imaging deviceto facilitate a reduction of optical crosstalk and radial fall-off inimages generated by the imaging device;

FIG. 6 shows a flow-chart of an example process that may be performed byan example image crosstalk analysis device of FIG. 5 to generate dataparameters specific to an example imaging device that may be used toreduce optical crosstalk and radial fall-off in images generated by theimaging device;

FIG. 7 shows a flow-chart of an example process that may be performed byan example imaging device to reduce optical crosstalk and radialfall-off in images generated by the imaging device based on the dataparameters generated by an image crosstalk analysis device using, forexample, the process described in FIG. 6;

FIG. 8 shows a flow-chart of an example adaptation of the processdescribed in FIG. 7 to reduce optical crosstalk and radial fall-off inBayer space formatted images;

FIG. 9 is a schematic diagram of example micro-lenses, color filters andCMOS sensors as may be used in the example optical sensing unitdescribed in FIG. 2;

FIG. 10 is an image representing optical crosstalk and radial fall-offin an example image generated by an example imaging device of aflat-field image with a uniform color temperature of 3000° Kelvin;

FIG. 11 is an image representing optical crosstalk and radial fall-offin an example image generated by an example imaging device of aflat-field image with a uniform color temperature of 5000° Kelvin;

FIG. 12 shows plots of example CMOS optical-to-electrical sensorresponses to different wavelengths of light by CMOS sensors at differentlocations in a CMOS sensor array relative to the distance to an imagecenter;

FIG. 13 shows plots of example blue, green, and red CMOSoptical-to-electrical sensor responses to the respective spectrums ofblue, green, and red light; and

FIG. 14 shows plots of the optical-to-electrical responses of the plotof FIG. 13, both before and after proposed sensor response sharpeningpost processing, relative to a plot representing the combined effect ofoptical crosstalk and radial fall-off gain on CMOS sensor responserelative the wavelength.

DETAILED DESCRIPTION OF EMBODIMENTS

FIG. 1 is a block diagram of an example imaging device that includes animage processing unit that may process raw image data generated by anarray sensor unit, e.g., a CMOS array sensor unit, to reduce opticalcrosstalk and radial fall-off in generated images. Embodiments of theexample imaging device, shown in FIG. 1, may be included in stand-alonedigital image cameras, as well as in digital image cameras embedded inother electronic devices, such as cellular phones, laptop computers andhand-held personal computing devices. As shown in FIG. 1, exampleimaging device 100 may include an optical sensing unit 102, an imageprocessing unit 104, a data storage unit 106 and an input/outputinterface 108.

FIG. 2 is a block diagram of an example embodiment of optical sensingunit 102, as shown in FIG. 1. As shown in FIG. 2, an example embodimentof optical sensing unit 102 may include a lens unit 202, a micro-lensarray unit 204, a color filter array unit 206, a CMOS array sensor unit208 and a data storage unit interface 210. In one example embodiment,the micro-lens array unit 204, color filter array unit 206, and CMOSarray sensor unit 208 may be formed as successive layers in a singleintegrated device in which a single micro-lens of micro-lens array unit204 is aligned with a single color filter element of color filter arrayunit 206, which is aligned with a single sensor of CMOS sensor arrayunit 208, as described below in greater detail with respect to FIG. 9.

For example, lens unit 202 may contain one or more fixed and/oradjustable focal length lenses capable of receiving light reflected froman object or scene to be photographed and focusing the light ontomicro-lens array unit 204. Micro-lens array unit 204 may contain anarray of micro-lenses, each respective micro-lens configured to receiveand direct unfiltered light from lens unit 202 in the direction of asingle colored filter element of color filter array unit 206 and asingle CMOS array sensor of CMOS array sensor unit 208. Color filterarray unit 206 may contain an array of colored filter elements, e.g., anarray of red, green and blue filter elements, each individual colorelement in the array aligned with a single micro-lens of micro-lensarray unit 204, so that light from a micro-lens is filtered by a singlecolored filter element before being passed, ideally, to a singleintended CMOS array sensor of CMOS array sensor unit 208. CMOS arraysensor unit 208 may include an array of CMOS light sensors, each sensoraligned with a colored filter element of color filter array unit 206 andcapable of generating an electrical response signal that correspondswith the intensity of filtered light incident on the CMOS sensor.

In operation, light incident on an individual micro-lens of micro-lensarray 204 may be directed through a single colored filter element ofcolor filter array unit 206, so that, in the ideal case, only lighthaving a wavelength passed by the single colored filter element isincident on a single intended CMOS sensor. One or more CMOS sensors maybe associated with an image pixel. In one example embodiment, threeindividual CMOS sensors, each configured to receive light via red, greenand blue color filter elements, respectively, may be associated with asingle image pixel, thereby allowing the pixel to capture any color inthe color spectrum based on the respective intensities of the respectivered, green and blue primary color elements contributing to the pixel.

Electrical responses generated by the respective CMOS sensors in CMOSsensor array 208 may be formatted by data storage unit interface 210 andstored in data storage unit 106 as raw image data files. These raw imagefiles may be retrieved via input/output interface 108 for viewing via adisplay integrated within the imaging device 100, or may be retrievedfor transfer to another device, such as a personal computer, forlong-term storage, viewing/printing and/or editing.

FIG. 3 is a block diagram of an example embodiment of image processingunit 104. As shown in FIG. 3, image processing unit 104 may include, animage processor unit controller 302, an electrical crosstalk correctionunit 303, a spectral crosstalk correction unit 304, an auto whitebalance unit 306, a color temperature approximation unit 308, a sensorsharpening unit 310, an adaptive radial fall-off curve unit 312, aspatial adaptive de-convolution unit 314 and a Bayer space adaptationunit 316. Raw image files may be retrieved by image processing unit 104from data storage unit 106 for processing that reduces the effect ofelectrical crosstalk, spectral crosstalk, optical crosstalk and/orradial fall-off errors in the raw image data. The revised image data maythen again be stored to data storage unit 106.

In operation, image processor unit controller 302 may maintain aworkflow state machine, and/or control parameters that allow each of therespective units described below to perform its assigned task. Forexample, image processor unit controller 302 may maintain a status listof which raw image data files stored in data storage unit 106 have beenprocessed to reduce the effect of electrical crosstalk, spectralcrosstalk, optical crosstalk and/or radial fall-off errors in the rawimage data. Further, as raw data associated with each image is processedby the respective units described below, image processor unit controller302 may receive status updates from the respective units so that imageprocessor unit controller 302 may coordinate the actions of therespective units in performing subsequent processing of the image data.

Example processes that may be executed by/coordinated by image processorunit controller 302 to reduce the effect of optical crosstalk and/orradial fall-off errors in the raw image data are described below withrespect to FIG. 7 and FIG. 8. The individual roles of the processingunits shown in FIG. 3 with respect to those processes are describedbelow.

Electrical crosstalk correction unit 303 may correct electricalcrosstalk in raw image data by applying one or more predeterminedelectrical crosstalk correction parameters to the stored image datasensor responses. As described above, electrical crosstalk results fromphoto-generated carriers having the possibility to move to neighboringcharge accumulation sites and is uniform across the entire image.Therefore, at the request of image processor unit controller 302,electrical crosstalk correction unit 303 may apply one or morepredetermined electrical crosstalk correction parameters to the storedimage data retrieved by image processor unit controller 302. Thepredetermined electrical crosstalk correction parameters may bedetermined using an example embodiment of an image crosstalk analysisdevice, as described below with respect to FIG. 5.

Spectral crosstalk correction unit 304 may correct spectral crosstalk inraw image data by applying a predetermined color correction matrix tothe ROB triads of each pixel in raw image data. As described above,spectral crosstalk may be due to imperfect color filter elements withincolor filter array unit 206 passing through some amount of unwantedwavelengths of light other than those wavelengths the filter is intendedto pass. Spectral crosstalk may cause the colors associated with the rawimage data to be unsaturated. Fortunately, spectral crosstalk isconsistent across images for the respective colors and may be correctedby applying color corrections based on a predetermined color correctionmatrix. Therefore, at the request of image processor unit controller302, spectral crosstalk correction unit 304 may apply a color correctionmatrix to a current set of raw image data retrieved by image processorunit controller 302. The predetermined color correction matrix may bedetermined using an example embodiment of an image crosstalk analysisdevice, as described below with respect to FIG. 5.

Auto white balance unit 306 may be used to identify a set of colortemperatures within image data. For example, auto white balance unit 306may automatically calculate the best white balance values according tothe lighting conditions identified by the imaging device user, e.g., viaa setting or selection via a user interface supported by the imagingdevice. Once a correct white balance is applied, auto white balance unit306 may identify a set of color temperatures in the auto white balancedimage data.

Color temperature approximation unit 308 may be used to match each colortemperature identified by auto white balance unit 306 within an image toa closest approximate temperature for which a predetermined sharpeningmatrix and adaptive radial fall-off coefficients, and/or specialadaptive de-convolution kernel coefficients have been generated andstored in data storage unit 106. Details related to the generation anduse of predetermined sharpening matrix and adaptive radial fall-offcoefficients, and/or special adaptive de-convolution kernel coefficientsare described below with respect to FIG. 5, FIG. 6, FIG. 7 and FIG. 8,and with respect to equations 8, 33-34, 30, and 26, respectively.

Sensor sharpening unit 310 may be used to apply a predeterminedsharpening matrix to a set of raw image data RGB pixel values to sharpenthe effective sensor response, as described below with respect to FIG. 7and FIG. 8, and with respect to equation 8. The predetermined sharpeningmatrix is developed for each of a selected set of color temperatures, asdescribed below with respect to FIG. 5 and equations 33 and 34. Aseparate predetermined sharpening matrix may be applied to image datapixels for each color temperature identified within the image.

Adaptive radial fall-off unit 312 may be used to apply a predeterminedset of adaptive radial fall-off curve coefficients to a set of sensorsharpened RGB pixel values to correct optical cross-talk and radialfall-off in the image data as described below with respect to FIG. 7 andFIG. 8, and with respect to equation 8. The predetermined adaptiveradial fall-off curve coefficients are developed for each of a selectedset of color temperatures, as described below with respect to FIG. 5 andequation 30. A set of adaptive radial fall-off curve coefficients may beapplied to a set of sensor sharpened RGB pixel values for each colortemperature identified within the image.

Spatial adaptive de-convolution unit 314 can provide an alternatetechnique for correcting optical cross-talk and/or radial fall-off. Apredetermined set of spatial adaptive de-convolution kernel coefficientsmay be applied to the raw image data in place of the sensor sharpeningmatrix and adaptive radial fall-off curve coefficients. For example, apredetermined set of spatial adaptive de-convolution kernelcoefficients, described below with respect to equation 26, may beapplied to the raw image data for each color temperature identified inthe image, as described below with respect to FIG. 7 and equation 24.Such an alternate technique for correcting optical cross-talk and/orradial fall-off in image data is mathematically derived from thecombination of linear form demosaicing and sensor response sharpening,and hence can operate on Bayer space directly. Furthermore, thede-convolution method is generalized by allowing more than 3 falloffcurves instead of one falloff curve for each channel of RGB formatimage.

Bayer space adaptation unit 316 may be used to reduce the effect ofspectral crosstalk, optical crosstalk and/or radial fall-off errors inthe raw image data generated and stored in Bayer space format. Asdescribed in greater detail below, the sensor sharpening matrix,adaptive radial fall-off curve coefficients, and spatial adaptivede-convolution kernel coefficients are configured for application to RGBformatted image data. As described below with respect to FIG. 4, Bayerspace adaptation unit 316 may provide additional processing capabilitiesto convert Bayer space formatted raw image from/to mosaic Bayer spaceformatted data to/from R, G₁, G₂, B formatted data so that Bayer spaceformatted image data may be processed using the above described sensorsharpening matrix, the adaptive radial fall-off curve coefficients,and/or the spatial adaptive de-convolution kernel coefficients, and thenconverted back to a Bayer space formatted image data.

FIG. 4 is a block diagram of an example embodiment of the Bayer spaceadaptation unit 316 described above with respect to FIG. 3. As shown inFIG. 4, an example embodiment of Bayer space adaptation unit 316 mayinclude a Bayer space adaptation controller 402, a de-mosaicing unit404, a de-sharpening unit 406 and Bayer space sampling unit 408.

Bayer space adaptation controller 402 may control operation ofde-mosaicing unit 404 to convert image data from Bayer space format toR, G₁, G₂, B format, may coordinate with image processor unit controller302 to coordinate use of electrical crosstalk correction unit 303,spectral crosstalk correction unit 304, auto white balance unit 306,color temperature approximation unit 308, sensor sharpening unit 310,adaptive radial fall-off curve unit 312, and spatial adaptivedc-convolution unit 314, as needed, to reduce the effect of spectralcrosstalk, optical crosstalk and/or radial fall-off errors in the rawimage data, and then to coordinate use of de-sharpening unit 406 andBayer sampling unit 408 to restore the revised image data to itsoriginal Bayer space format.

De-mosaicing unit 404 may be invoked by Bayer space adaptationcontroller 402 to reconstruct, or de-mosaic Bayer formatted image data.For example, de-mosaicing unit 404 may apply interpolation algorithms,such as bilinear interpolation, in which the red value of a non-redpixel is computed as the average of the two or four adjacent red pixels,and similarly for blue and green. Other linear or edge adaptivedemosaicing techniques, such as bicubic interpolation, splineinterpolation, Adam-Halmilton demosacing, and the like, may also beused.

De-sharpening unit 406 may be used to remove the sensor sharpeningapplied by sensor sharpening unit 310, once the effects of electricalcrosstalk, spectral crosstalk, optical crosstalk and/or radial fall-offerrors have been corrected for in the raw image data. For example,de-sharpening unit 406 may apply a de-sharpening matrix, i.e., theinverse matrix of the sensor spectrum response sharpening matrix, toremove the sensor sharpening applied by sensor sharpening unit 310.Unless de-sharpened, the sharpened R, G, B values may impact the edgedetection of demosaicing algorithm and may result in an image with sharpedges and big noise, which may degrade image quality.

Bayer space sampling unit 408 may be used to sub-sample the de-sharpenedR, G, B image data, in which the effects of electrical crosstalk,spectral crosstalk, optical crosstalk and/or radial fall-off errors havebeen corrected. In this manner, the corrected image data may be returnedto the Bayer space format in which the image was originally stored indata storage unit 106.

FIG. 5 is a block diagram of an example image crosstalk analysis devicethat may be used to analyze images generated by an example imagingdevice to facilitate a reduction of electrical crosstalk, spectralcrosstalk, optical crosstalk, electrical crosstalk and/or radialfall-off in images generated by the imaging device. Image crosstalkanalysis device may receive image data from an imaging device, e.g., viainput/output interface 108 of example imaging device 100, as describedabove with respect to FIG. 1, and may analyze the image data to produce,for example, color correction tables, electrical crosstalk correctionparameters, sharpening matrices, de-sharpening matrices, adaptive radialfalloff curve correction parameters and/or spatial adaptivede-convolution kernel coefficients. The generated correction data may betransferred to and stored in data storage unit 106, e.g., viainput/output interface 108 of example imaging device 100. As describedbelow, image processing unit 104 may selectively retrieve and use thestored, color correction tables, electrical crosstalk correctionparameters, sharpening matrices, de-sharpening matrices, adaptive radialfalloff curve correction parameters and/or spatial adaptivede-convolution kernel coefficients to make corrections to a stored imagegenerated by optical sensing unit 102.

As shown in FIG. 5, an example embodiment of image crosstalk analysisdevice 500 may include a computer system data bus 502 that allows aprocessor 504 to communicate with, and exchange information withhardware components of the image crosstalk analysis device such as: avolatile memory storage 506, which allows the processor 504 to storeprogram instructions in local memory for execution and to store andmaintain temporary data necessary for execution of the stored programinstructions; a non-volatile memory storage 508, such as a hard-driveand/or firmware storage, which allows processor 504 to access, retrieveand/or store data and program instructions for later use/execution bythe processor; a local display 510, which may support a visual interfacewith a crosstalk analysis technician, or user, who is responsible forassuring that generated crosstalk correction data files meet the user'sexpectation requirements; a control panel/keyboard 512, and/or a cursorcontrol device that allow the processor to receive user instructionsand/or information and/or feedback; an input/output interface 514capable of supporting the receipt of image data files and image devicedescription data from an imaging device, e.g., such as imaging device100 described above with respect to FIG. 1, and capable of supportingthe transfer of correction parameter data files generated by imagecrosstalk analysis device 500 to imaging device 100; and a flat-fieldgenerator capable of displaying flat field images corresponding toselected color temperatures that may be photographed by imaging device100 to generate corresponding color temperature flat-field image datawhich may be transferred from imaging device 100 to image crosstalkanalysis device 500 via input/output interface 514.

As further shown in FIG. 5, processor 504 may include internalcomponents that allow processor 504 to communicate with theabove-described hardware components to send and receive data andinstructions over system bus 502. Such components may include: aninput/output unit 520 that manages communication exchanges betweenprocesses executed by processor 504 and the system bus 502; and a datamanagement unit 522, which allows processor 504 to maintain a local setof control parameters such as counters, pointers, and segments ofexecutable program instructions for execution.

For example, when provided with executable instructions, processor 504may, in accordance with instructions/commands received from a user viacontrol panel/keyboard 512, retrieve and initiate controlparameters/pointers for the execution of program instructions related tothe creation of crosstalk correction data files for a specific imagingdevice 100. For example, at startup, processor 504 may retrieve and loadprogram instructions from non-volatile memory storage 508 into volatilememory storage 506 for execution and may maintain control parameters indata management unit 522 for use in controlling the simultaneous and/orsequential execution of the program instructions retrieved forexecution.

For example, as shown in FIG. 5, processor 504 may establish, based onstored program instructions retrieved for execution from non-volatilememory storage 508: a user interface unit 526 that supports a userinterface between the image crosstalk analysis device and a crosstalkanalysis technician, or user, via local display 510 and controlpanel/keyboard 512, e.g., to receive user commands and instructions formanaging the creation of crosstalk correction data files for a specificimaging device; a crosstalk analysis controller 524 that coordinates thecrosstalk analysis process and the generation of crosstalk correctiondata files for an imaging device; a spectral crosstalk correction unit528 that supports the development of color correction tables to reducethe effect of spectral crosstalk in image data generated by an imagingdevice; an electrical crosstalk correction unit 530 that supports thedevelopment of a uniform correction for electrical crosstalk in imagedata generated by an imaging device; a flat-field control unit 532 thatcontrols the generation of color temperature flat-field data files; achannel filtration unit 534 that filters noise and dark current fromdata image files received from an imaging device; a sharpening matrixgeneration unit 536 that supports generation of sensor sharpeningmatrices; a radial fall-off curve generation unit 538 that supports thegeneration of radial fall-off correction parameters; and ade-convolution kernel coefficient generation unit 540 that supports thegeneration of de-convolution kernel coefficients.

In operation, each of the above-described modules/controllers executedby processor 504 may maintain a workflow state machine, and/or controlparameters that allow each module to perform its assigned task. Forexample, crosstalk analysis controller 524 may maintain a work-flowstate machine, and/or control parameters that allows crosstalk analysiscontroller 524 to coordinate the functions performed by other unitsexecuted by processor 504 to perform crosstalk analysis and generatecrosstalk correction data files for an imaging device.

For example, crosstalk analysis controller 524 may contain a set ofcontrol parameters that allows it to initiate the performance ofspecific actions by each of the other modules executed by processor 504.Each module executing an assigned task may provide crosstalk analysiscontroller 524 with status updates, e.g., indicating the completion ofan assigned task, thereby allowing crosstalk analysis controller 524 toorchestrate activities performed by the respective modules/controllersin series and/or in parallel to expedite the smooth and efficientexecution of a crosstalk analysis and crosstalk correction datageneration session with a user.

User interface unit 526 may generate and present to a user via, forexample, local display 510, displays that allow the user to interactwith and control a crosstalk analysis and crosstalk correction datageneration session. For example, at the start of a crosstalk analysisand crosstalk correction data generation session, a user may bepresented with a display that allows the user to connect to an imagingdevice 100 that may be physically connected to image crosstalk analysisdevice 500, e.g., via input/output interface 514. Throughout thecrosstalk analysis and crosstalk correction data generation session,user interface unit 526 may guide the user through the respectiveprocesses that require user input/feedback based on an overall processflow controlled by crosstalk analysis controller 524.

Electrical crosstalk correction unit 530 may generate electricalcrosstalk correction parameters that maybe uploaded to imaging device100 and applied by image processing unit 104 to future images generatedby imaging device 100. As described above, electrical crosstalk resultsfrom photo-generated carriers having the possibility to move toneighboring charge accumulation sites and is uniform for the wholeimage. Electrical crosstalk correction unit 530 may compare pixel valuesfrom flat-field images generated by an imaging device, with a set ofreference pixel values for reference flat-field images for the samerespective color temperatures stored within non-volatile memory storage508 and may generate one or more electrical crosstalk correctionparameters for the imaging device. User interface unit 526 may be usedto provide an opportunity to visually view comparisons of the imagingdevice's respective flat-field images with corresponding referenceflat-field images both before and after application of the proposedelectrical crosstalk correction parameters. The user may modify the oneor more proposed electrical crosstalk correction parameters andrepeatedly compare the effect of the one or more proposed/revisedelectrical crosstalk correction parameters on the respective imagingdevice flat-field images in comparison with the reference images untilthe user's image quality expectations are satisfied. Electricalcrosstalk correction parameters approved by the user may be stored forsubsequent upload to imaging device 100 for application by imageprocessing unit 104 to future images generated by imaging device 100.

Spectral crosstalk correction unit 528 may generate color correctiontables that maybe uploaded to imaging device 100 and applied by imageprocessing unit 104 to future images generated by imaging device 100 tocounter spectral crosstalk identified by spectral crosstalk correctionunit 528 in flat-field images generated by imaging device 100. Asdescribed above, spectral crosstalk results from imperfect color filterspassing through some amount of unwanted light of other colors and maycause unsaturated colors in a generated image. Spectral crosstalk may becorrected with one or more color correction matrices. Spectral crosstalkcorrection unit 528 may compare pixel values from flat-field imagesgenerated by imaging device 100, with a set of reference pixel valuesfor reference flat-field images for the same respective colortemperatures stored within non-volatile memory storage 508 and maygenerate one or more color correction tables for the imaging device.User interface unit 526 may be used to provide an opportunity tovisually view comparisons of the imaging device's respective flat-fieldimages with corresponding reference flat-field images both before andafter application of the proposed color correction tables. The user maymodify the one or more proposed color correction tables and repeatedlycompare the effect of the one or more proposed/revised color correctiontables on the respective imaging device flat-field images in comparisonwith the reference images until the user's image quality expectationsare satisfied. Color correction tables approved by the user may bestored for subsequent upload to imaging device 100 for application byimage processing unit 104 to future images generated by imaging device100.

Flat-field control unit 532 may coordinate the generation of flat-fieldimages for a set of color temperatures selected by image crosstalkanalysis device 500 for use in a crosstalk analysis and crosstalkcorrection data generation session. For example, flat-field control unit532, via crosstalk analysis controller 524 may instruct flat-fieldgenerator 516 to generate a series of flat-field images for a range ofcolor temperatures. Upon presentation of each respective flat-fieldimage, flat-field control unit 532, via crosstalk analysis controller524 and input/output interface 514 may instruct imaging device 100 tophotograph each respective flat-field image and transfer the generatedflat-field image data to image crosstalk analysis device 500 for storagein association with the respective color temperature and a uniqueidentifier/descriptor of the imaging device. These stored flat-fieldimages maybe used to support subsequent crosstalk analysis and crosstalkcorrection data generation performed during the session, as described ingreater detail below.

Channel filtration unit 534 may be configured to filter image datareceived by image crosstalk analysis device 500 and filter out noise anddark current, i.e., repetitive fixed pattern noise, detected in theimage data. Such noise in the image data may be corrected with one ormore digital filters and/or smoothing functions. Channel filtration unit534 may compare pixel values from flat-field images generated by imagingdevice 100, with a set of reference pixel values for referenceflat-field images for the same respective color temperatures storedwithin non-volatile memory storage 508 and may recommend one or moredigital filters and/or smoothing functions. User interface unit 526 maybe used to provide an opportunity to visually view comparisons of theimaging device's respective flat-field images with correspondingreference flat-field images both before and after application of theproposed digital filters and/or smoothing functions. The user may selectone or more of the proposed digital filters and/or smoothing functionsand repeatedly compare the effect of the one or more proposed digitalfilters and/or smoothing functions on the respective imaging deviceflat-field images in comparison with the reference images until theuser's image quality expectations are satisfied. Digital filters and/orsmoothing functions approved by the user may be stored for subsequentupload to imaging device 100 for application by image processing unit104 to future images generated by imaging device 100.

Sharpening matrix generation unit 536 may generate, for each colortemperature flat-field image generated for imaging device 100, asharpening matrix as described in greater detail below with respect toequations 33 and 34. The generated sharpening matrices may be uploadedto imaging device 100 and may be applied by image processing unit 104 tofuture images generated by imaging device 100 as described above withrespect to FIG. 3 and FIG. 4, and as described below with respect toFIG. 7 and FIG. 8.

Radial fall-off curve generation unit 538 may generate, for each colortemperature flat-field image generated for imaging device 100, radialfall-off coefficients as described in greater detail below with respectto equation 30. The generated radial fall-off coefficients may beuploaded to imaging device 100 and may be applied by image processingunit 104 to future images generated by imaging device 100 as describedabove with respect to FIG. 3 and FIG. 4, and as described below withrespect to FIG. 7 and FIG. 8.

De-convolution kernel coefficient generation unit 540 may generate, foreach color temperature flat-field image generated for imaging device100, de-convolution kernel coefficients, as described in greater detailbelow with respect to equation 26. The generated radial fall-offcoefficients may be uploaded to imaging device 100 and may be applied byimage processing unit 104 to future images generated by imaging device100, as described below with respect to FIG. 7 and FIG. 8.De-convolution kernel coefficients may be applied to the respectivecolor temperatures identified in an image to correct for opticalcrosstalk and radial falloff in an image generated by an imaging devicewhich has been determined to have sufficiently sharp sensor sensitivityso as to not require use of a sharpening matrix. De-convolution kernelcoefficients may be applied to image data for a color temperature inplace of the sharpening matrix and radial fall-off curve coefficients.For example, user interface unit 526 may be used to provide a user withan opportunity to visually view comparisons of an image generated by animaging device corrected with first the sharpening matrix and radialfall-off curve coefficients generated by sharpening matrix generationunit 536 and radial fall-off generation unit 538, respectively, followedby the same image corrected with de-convolution kernel coefficientsgenerated by de-convolution kernel coefficient generation unit 540. Inthis manner, a user may select which approach is applied by imagingdevice 100 to future generated image data.

FIG. 6 shows a flow-chart of an example process that may be performed byan example image crosstalk analysis device 500 of FIG. 5. The processmay be used to generate correction parameters for imaging device 100,e.g., sharpening matrices, radial fall-off curve coefficients, andde-convolution kernel coefficients, that may be uploaded to imagingdevice 100 for application by image processing unit 104 to correctoptical crosstalk and radial falloff in future images generated byimaging device 100. As shown in FIG. 6, operation of process 600 beginsat step S602 and proceeds to step S604.

In step S604, crosstalk analysis controller 524 may select, e.g., basedon technical description data received from an attached imaging device100, such as CMOS sensor array density and/or raw image data format, anumber of neighbor pixels (N) and a number of fall-off curves (M), andoperation of the process continues to step S606.

In step S606, flat-field control unit 532 may coordinate the operationof flat-field generator 516 and imaging device 100 connected to imagecrosstalk analysis device 500 via input/output interface 514, asdescribed above with respect to FIG. 5, to generate and download fromimaging device 100 flat-field data images generated for a set of colortemperatures, and operation of the process continues to step S608.

In step S608, channel filtration unit 534 maybe invoked to filter noiseand dark current from the generated flat-field images, as describedabove with respect to FIG. 5, and operation of the process continues tostep S610.

In step S610, crosstalk analysis controller 524 may select a first/nextcolor temperature in the flat-field image generated and filtered in stepS606 and step S608, respectively, and operation of the process continuesto step S612.

In step S612, crosstalk analysis controller 524 determines if number ofneighbor pixels (N) is equal to the number of fall-off curves (M). Ifso, operation of the process continues to step S614, otherwise,operation of the process continues to step S620.

In step S614, sharpening matrix generation unit 536 may initialize asharpening matrix to the identity matrix, and operation of the processcontinues to step S616.

In step S616, sharpening matrix generation unit 536 may minimize thetotal error for pixels of the selected flat-field image for eachchannel, e.g., R, G, B, as described below with respect to equations 33and 34, e.g., using a method of steepest gradient descent, resulting inthe generation of a sharpening matrix W_(M×N), and operation of theprocess continues to step S618.

In step S618, the generated sharpening matrix W_(M×N) may be stored inassociation with the identifiers, e.g., make, model, and serial number,of the attached imaging device 100, and operation of the processcontinues to step S620.

In step S620, crosstalk analysis controller 524 determines if the numberof neighbor pixels (N) is not equal to the number of fall-off curves(M). If so, then operation of the process continues to step S622;otherwise, operation of the process continues to step S624.

In step S622, sharpening matrix generation unit 536 may minimize thetotal error for pixels of the selected flat-field image for eachchannel, e.g., R, G, B, as described below with respect to equations 33and 34, e.g., using a method of steepest gradient descent, resulting inthe generation of a sharpening matrix W_(M×N), and operation of theprocess continues to step S624.

In step S624, radial fall-off curve generation unit 538 may generateradial fall-off coefficients for the selected flat-field image for eachchannel, e.g., R, G, B, as described below with respect to equation 30,and the resulting radial fall-off coefficients may be stored inassociation with the identifiers, e.g., make, model, and serial number,of the attached imaging device 100, and operation of the processcontinues to step S626.

In step S626, the de-convolution kernel coefficient generation unit 540may generate spatial adaptive de-convolution kernel coefficients for theselected flat-field image for each channel, e.g., R, G, B, as describedbelow with respect to equation 26, and the resulting spatial adaptivedc-convolution kernel coefficients may be stored in association withidentifiers, e.g., make, model, and serial number, for the attachedimaging device 100, and operation of the process continues to step S628.

In step S628, crosstalk analysis controller 524 determines if the lastflat-field image has been selected. If so, operation of the processcontinues to step S630 and operation of the process terminates;otherwise, operation of the process continues to step S610.

As described in the above process flow with respect to steps S610through S618, a sharpening matrix W_(M×N) may be generated for aselected set of flat-field images, e.g., a set of flat-field images thatincludes a 3000K temperature flat-field image and a 5400K temperatureflat-field image, using a method of steepest gradient descent.

For example, if an initial sharpening matrix is not accurately selected,e.g., such as a sharpening matrix initialized to the identity matrix instep S614, above, the radial falloff gain calculated based on a 3000Kimage and calculated based on a 5400K image will be quite different.However, as the sharpening matrix is optimized, the radial falloff gaindifference between the 3000K and 5400K will be minimized as well.

Example steps for calculating a falloff gain difference between a 3000Kimage and a 5400K image may include:

-   -   1. Performing sensor response sharpening using the current        sensor response sharpening matrix;    -   2. Calculating the falloff gain based on the sharpened RGB        image: if the pixel ROB values at the image center is selected        as the reference (R_(ref), G_(ref), B_(ref)), then the gain        value at pixel (x,y) can be calculated as Gain_(R)(x,y)        R_(ref)/R(x,y), Gain_(G)(x,y)=G_(raf)/(x,y) and Gain_(B)(x,y)        B_(ref)/B(x,y); and    -   3. Calculating the gain difference sum of all pixels:        error_(R)=Σ|Gain_(R,3000K)(x,y)−Gain_(R,5400K)(x,y)|,        error_(G)=Σ|Gain_(G,3000K)(x,y)−Gain_(G,5400K)(x,y)| and        error_(B)=Σ|Gain_(E,3000K)(x,y)−Gain_(B,5400K)(x,y)|.

Because adjusting each row of the current sensor response sharpeningmatrix will change the value of error_(R), error_(G) and error_(B)correspondingly, the method of steepest descendent can be used tominimize all the 3 errors.

For example, flat-field color temperature images may be generated, asdescribed above in step S606 and may be filtered as described above withrespect to step S608 to remove noise and dark current from the generatedflat-field images. Further, depending on the nature of the imagingdevice that generated the flat field images, the flat field images maybe converted, e.g., from Bayer space to RGB, and the images may besub-sampled to a lower resolution to reduce the amount of calculation.

Example steps for applying a method of steepest gradient descent in stepS616 may include:

-   -   1. Initializing the sharpening matrix to the unity matrix (done        in step S614);    -   2. Selecting a small value δ as the initial step length;    -   3. Calculating the descendent direction v;    -   4. Calculating the new matrix by subtracting the matrix with δ*        D and the new gain ratio variance;    -   5. If the new gain ratio variance is smaller than the old gain        ratio variance, the sharpening matrix is replaced with the step        adjusted sharpening matrix, and the process returns to step 4;    -   6. If the new gain ratio variance is bigger than the old gain        ratio variance, then the step length is decreased, e.g.,        δ←δ*0.8;    -   7. If δ is bigger than a given threshold (for example 0.0001),        the process returns to step 4, otherwise the process is        completed.

For a given sharpening matrix, the steps to calculate the direction ofsteepest descendent, as described above, may include:

-   -   1. Calculating a gain ratio variance k0 for the given sharpening        matrix;    -   2. For each coefficient in the 3×3 matrix, changing the value by        Δ and calculating the gain ratio variance., i.e., 9 new gain        ratio variance k1˜k9;    -   3. Estimating the partial derivative to each matrix coefficient,        i.e., dk(i)=(k1−k0)/delta; and

Calculating the direction of steepest descendent, e.g., the direction ofsteepest descendent, v, is given by matrix D=[dk(1), dk(2), dk(3) dk(4),dk(5), dk(6) dk(7),dk(8), dk(9)] sqrt[Σ(dk(i)^2)].

FIG. 7 shows a flow-chart of an example process that may be performed byan example imaging device 100 to reduce optical crosstalk and radialfall-off in images generated by the imaging device based on the imagecorrection parameters generated by an image crosstalk analysis device500 using, for example, the process described in FIG. 6. The processdescribed with respect to FIG. 7, assumes that sharpening matricesW_(M×N), radial fall-off coefficients, and de-convolution kernelcoefficients generated for an imaging device in the process describedabove with respect to FIG. 6, have been uploaded to and stored withinthe imaging device and are accessible to image processing unit 104within imaging device 100, as described above with respect to FIG. 1.Further, the process assumes that electrical crosstalk corrections andspectral crosstalk corrections performed by electrical crosstalkcorrection unit 303 and spectral crosstalk correction unit 304, asdescribed above with respect to FIG. 3, are applied in a separateprocess from the process described below with respect to FIG. 7 toreduce optical crosstalk and radial falloff. As shown in FIG. 7,operation of process 700 begins at step S702 and proceeds to step S704.

In step S704, image processing unit 104 within imaging device 100 mayretrieve image data generated by optical sensing unit 102 and stored indata storage unit 106, as described above with respect to FIG. 1, andoperation of the process continues to step S706.

In step S706, auto white balance unit 306 may perform color balancing asdescribed above with respect to FIG. 3, and may detect a set of colortemperatures in the color balanced image data, and operation of theprocess continues to step S708.

In step S708, image processor unit controller 302 may select afirst/next color temperature, and operation of the process continues tostep S710.

In step S710, color temperature approximation unit 308 may determine aclosest matching color temperature to the selected color temperature forwhich image correction parameters are stored in data storage unit 106 ofimaging device 100, e.g., using interpolation techniques as describedabove with respect to FIG. 3, and operation of the process continues tostep S712.

In step S712, image processor unit controller 302 may retrieve storedradial fall-off coefficients for each image data channel, e.g., R, G, B,for the selected color temperature and operation of the processcontinues to step S714.

In step S714, image processor unit controller 302 may retrieve storedsharpening matrix (W_(M×N)) for each image data channel, e.g., R, G, B,for the selected color temperature and operation of the processcontinues to step S716.

In step S716, image processor unit controller 302 may instructsharpening matrix generation unit 310 to apply the sharpening matrix(W_(M×N)) retrieved for each image data channel for the currentlyselected color temperature to the image data retrieved in step S704, andmay instruct adaptive radial falloff curve unit 312 to next apply theretrieved radial falloff coefficients for each image data channel forthe currently selected color temperature to the sensor sharpened imagedata, as described below with respect to equation 8, and operation ofthe process continues to step S718.

In step S718, image processor unit controller 302 determines if the lastcolor temperature in the image has been selected. If so, operation ofthe process continues to step S720; otherwise, operation of the processcontinues to step S708.

In step S720, image processor unit controller 302 may store the opticalcrosstalk/radial fall-off corrected image data to data storage unit 106,and operation of the process continues to step S722 and terminates.

It is noted that if a user has determined that sensor sharpening is notrequired for a particular embodiment of imaging device 100, for example,based on the user's comparative assessment of optical crosstalk andradial falloff corrected images, as described above with respect FIG. 5,image processor unit controller 302 may instruct spatial adaptivede-convolution unit 314 to apply, in accordance with equation 24described below, the de-convolution kernel coefficients generated byde-convolution kernel coefficient generating unit 540 in place of thesharpening matrix (W_(M×N)) and radial falloff coefficients as describedabove at step S716, and operation of the process continues to step S718.

FIG. 8 shows a flow-chart of an example adaptation of the processdescribed above with respect to FIG. 7. The process described below withrespect to FIG. 8, adapts the process described above with respect toFIG. 7 to reduce optical crosstalk and radial fall-off in Bayer spaceformatted images. As shown in FIG. 8, operation of process 800 begins atstep S802 and proceeds to step S804.

In step S804, image processing unit 104 within imaging device 100 mayretrieve Bayer space formatted image data generated by optical sensingunit 102 and stored in data storage unit 106, as described above withrespect to FIG. 1, and operation of the process continues to step S806.

In step S806, image processor unit controller 302 may invoke Bayer spaceadaptation unit 316 which, via Bayer space adaptation controller 402 andde-mosaicing unit 404, may demosaic the Bayer space image data, asdescribed above with respect to FIG. 4, and operation of the processcontinues to step S808.

In step S808, image processor unit controller 302 may reduce opticalcrosstalk and radial fall-off in the R, G₁, G₂, B formatted image datausing the process described above with respect to FIG. 7, and operationof the process continues to step S810.

In step S810, image processor unit controller 302 may invoke Bayer spaceadaptation controller 402 and de-sharpening unit 406, to de-sharpen thesharpen image data as described above with respect to FIG. 4, andoperation of the process continues to step S812.

In step S812, image processor unit controller 302 may invoke Bayer spaceadaptation controller 402 and Bayer space sampling unit 408, tosub-sample the R, G₁, G₂, B formatted image data back into Bayer spaceformatted image data, and operation of the process continues to stepS814.

In step S814, image processor unit controller 302 may store the Bayerspace formatted image data to data storage unit 106, and operation ofthe process continues to step S816 and terminates.

FIG. 9 is a schematic diagram of example micro-lens array unit, colorfilter array unit, and CMOS array sensor unit, as described above withrespect to FIG. 2. As shown in FIG. 9, the micro-lens array unit, colorfilter array unit, and CMOS array sensor unit may be formed assuccessive layers in a single integrated device. As further shown inFIG. 9, individual micro-lenses, e.g., as shown at 902 and 904, in themicro-lens array unit, may be aligned with individual color filter arrayelements, e.g., as shown at 906 and 908, in the color filter array unit,which may be further aligned with individual CMOS sensors, e.g., asshown at 914 and 916, in the CMOS sensor array unit.

In operation, unfiltered light 918 focused by a lens unit (not shown)onto a micro-lens 904 of the micro-lens array unit may be redirected bythe micro-lens through a color filter element 908 aligned with themicro-lens in the direction of a CMOS sensor aligned with themicro-lens. However, as shown in FIG. 9, unfiltered light 918 impactsthe respective micro-lenses at greater angles of incidence the furtherthe respective micro-lenses are from the lens'center of focus. As aresult, longer wavelengths of light within the unfiltered light incidentupon micro-lenses relatively far from the lens' center of focus may notbe sufficiently redirected. For example, as shown in FIG. 9, light 910filtered by red color filter element 906 which has not been sufficientlyredirected in the direction of an intended CMOS sensor, may instead landon neighboring CMOS sensor 916 causing that sensor to registerinappropriately high sensor response. Due to the alternating patterns ofred and green and green and blue in adjacent rows of CMOS sensor arrays,the green CMOS sensors may register inappropriately high sensorresponses and the respective red CMOS sensors may register unusually lowvalues, resulting in raw image data that contains brighter pixels, i.e.,increased red tones, towards the center of the image and darker pixelvalues, i.e., increased blue and green tones, in a direction away fromthe center and towards an edge of the image,

FIG. 10 is an image representing optical crosstalk and radial fall-offin an example image generated by an example imaging device of aflat-field image with a uniform color temperature of 3000° Kelvin. FIG.11 is an image representing optical crosstalk and radial fall-off in anexample image generated by an example imaging device of a flat-fieldimage with a uniform color temperature of 5000° Kelvin. As shown in FIG.10 and FIG. 11, different color temperatures exhibit different opticalcrosstalk and radial fall-off conditions. As a result, it is difficultto correct for color non-uniformity in an image that includes largeareas of color with very different color temperatures.

For example, a CMOS sensor's input light spectrum not only depends onthe illuminant's color temperature, but also on a photographed object'sreflection spectrum. For example, on a sunny day, white objects, such ashighly reflective sand, may have a color temperature of 7000K, however,a clear blue sky in the same image may have a color temperature closerto 12000K. So, if a radial fall-off curve corresponding to 7000K isapplied to the image data, color non-uniformity will be observed in theblue of the sky. The same problem will be present with respect to everyobject in the image with a color temperature significantly differentfrom the color temperature of the flat field board used to capture theflat field image.

There are at least three problems associated with the use of existingradial fall-off curve techniques. First the existing approaches do notaddress optical crosstalk which increases the color error and is notcorrected for by existing radial fall-off curve techniques. Second,existing radial fall-off curve techniques depend on auto-white balanceto provide correct color temperature information. But existingauto-white balance techniques depend on an image free from chromauniformity problems, therefore, under conditions in which chromauniformity are present, auto-white balance performance may be degradedand/or wrong. Third, to eliminate visible chroma non-uniformity, 4 to 5radial fall-off curve gain tables may be required to address allpossible color temperatures within an image.

In the equations derived below, the function L(λ,x, y) is used to denotelight received at an imaging device, either directly from a light sourceor reflected from an object or scene to be photographed, where λ is thewavelength and (x,y) is the pixel position. Therefore, the function L(λ,x, y) may represent the light information of the real scene to becaptured, and the response of the function L(λ, x, y) may changesignificantly for different wavelengths of light at different x/ylocations on the CMOS array sensor on which the light falls.

The function F(λ, x, y) is used to denote CMOS sensor array optical toelectrical sensor response to incoming light under the combined effectof optical crosstalk and radial fall-off gain. As indicated by itsrespective operand symbols the response F(λ,x,y) is a function ofwavelength and pixel position.

FIG. 12 shows plots of the F(λ, x, y) optical-to-electrical sensorresponses to different wavelengths of light by CMOS sensors at differentlocations in a CMOS sensor array relative to the distance to an imagecenter. As shown in FIG. 12, F(λ, x, y) is not constant, but changesslowly as shown by the plots of F(λ, x, y) for blue, green, yellow,orange and red at plots 1202, 1204, 1206, 1208 and 1210, respectively.The F(λ, x, y) response has larger values at the image center, whereradial fall-off and optical crosstalk are both low, but the F(λ, x, y)response has lower values toward the surrounding edges of the image, asradial fall-off and optical crosstalk both increase. As shown in FIG.12, the gain reduces faster, relative to the distance of a CMOS sensorfrom the center of the image, for longer wavelengths of light (e.g., redlight at plot 1210) than for shorter wavelengths of light, e.g., bluelight at plot 1202.

The functions S_(R)(λ), S_(G)(λ) and S_(B)(λ) are used to denote thesensor sensitivity of the respective red, green and blue CMOS sensorsrelative to the wavelength of light received. FIG. 13 shows plots ofexample blue, green, and red CMOS optical-to-electrical sensorresponses, i.e., S_(B)(λ), S_(G)(λ) and S_(R)(λ), to the respectivespectrums of blue, green, and red light. As shown in FIG. 13, the sensorresponses are fairly rounded with gentle slopes, resulting insignificant overlap with respect to the respective wavelengths to whicheach sensor type produces a response. Such rounded peaks arerepresentative of most sensor responses and, therefore, without colorcorrection, colors in the images generated by imaging devices thatinclude such sensors are likely to not have sufficient saturation ascompared to the original photographed scene.

Based on the description provided above, the sensor output R, G, B,values for R, G, B, pixels in a CMOS array sensor unit may be denoted byequation 1.

$\begin{matrix}\left\{ \begin{matrix}{{R\left( {x,y} \right)} = {\int{{L\left( {\lambda,x,y} \right)}{F\left( {\lambda,x,y} \right)}{S_{R}(\lambda)}{\mathbb{d}\lambda}}}} \\{{G\left( {x,y} \right)} = {\int{{L\left( {\lambda,x,y} \right)}{F\left( {\lambda,x,y} \right)}{S_{G}(\lambda)}{\mathbb{d}\lambda}}}} \\{{B\left( {x,y} \right)} = {\int{{L\left( {\lambda,x,y} \right)}{F\left( {\lambda,x,y} \right)}{S_{B}(\lambda)}{\mathbb{d}\lambda}}}}\end{matrix} \right. & {{EQ}.\mspace{14mu} 1}\end{matrix}$

Where L(λ, x, y) is the incident light received at an imaging device;

F(λ, x, y) is sensor response under the combined effect of opticalcrosstalk and radial fall-off; and

S_(R)(λ), S_(G)(λ) and S_(B)(λ) are the sensor sensitivity to red, greenand blue light.

The existing radial fall-off curve approach, which is described above asinadequate for addressing the effects of radial fall-off in images withlarge areas of colors with significant differences in color temperature,and inadequate in addressing the combined effect of both radial fall-offand optical crosstalk, may be denoted by equation 2.

$\begin{matrix}{\quad\begin{matrix}\left\{ \begin{matrix}{{{R\left( {x,y} \right)} \approx {{F\left( {\lambda_{R},x,y} \right)}{\int{{L\left( {\lambda,x,y} \right)}{S_{R}(\lambda)}{\mathbb{d}\lambda}}}}} = {{F_{R}\left( {x,y} \right)} \cdot {R_{desired}\left( {x,y} \right)}}} \\{{{G\left( {x,y} \right)} \approx {{F\left( {\lambda_{G},x,y} \right)}{\int{{L\left( {\lambda,x,y} \right)}{S_{G}(\lambda)}{\mathbb{d}\lambda}}}}} = {{F_{G}\left( {x,y} \right)} \cdot {G_{desired}\left( {x,y} \right)}}} \\{{{B\left( {x,y} \right)} \approx {{F\left( {\lambda_{B},x,y} \right)}{\int{{L\left( {\lambda,x,y} \right)}{S_{B}(\lambda)}{\mathbb{d}\lambda}}}}} = {{F_{B}\left( {x,y} \right)} \cdot {B_{desired}\left( {x,y} \right)}}}\end{matrix} \right. & \;\end{matrix}} & {{EQ}.\mspace{14mu} 2}\end{matrix}$

Where R(x,y), G(x,y) and B(x,y) represent the stored ROB values for eachpixel;

F(λ_(R/G/B), x, y) are the radial fall-off curve values for red, greenand blue color temperatures, respectively;

F_(R/G/B)(x, y) are approximations for the respective radial fall-ofcurve values; and

R/G/B_(desired)(x,y) represent the actual, or true, red, green and bluevalues at the respective pixels.

The approximation presented in equation 2 is often inadequate in that itis unable to handle scenes with relative large areas of colors withsignificantly different color temperatures. However, the approachpresented in equation 2 would be greatly improved if the color filtershad a sharp, impulse response, in which the respective sensor responseshave a constant value for designated red, green and blue wavelengths,and a value of zero for other wavelengths, as depicted below in equation3.

$\begin{matrix}{{S_{R}(\lambda)} = {{\quad\quad}\left\{ {\begin{matrix}{K_{R},{\lambda = \lambda_{R}}} \\{0,{\lambda \neq \lambda_{R}}}\end{matrix},{{S_{G}(\lambda)} = \left\{ {{\begin{matrix}{K_{G},{\lambda = \lambda_{G}}} \\{0,{\lambda \neq \lambda_{G}}}\end{matrix}{and}\mspace{14mu}{S_{B}(\lambda)}} = \left\{ \begin{matrix}{K_{B},{\lambda = \lambda_{B}}} \\{0,{\lambda \neq \lambda_{B}}}\end{matrix} \right.} \right.}} \right.}} & {{EQ}.\mspace{14mu} 3}\end{matrix}$

Where K_(R/G/B) are constant values.

In actuality, sensors with such impulse responses are not available,however, linear processing may be used to sharpen the sensor response asmuch as possible, thereby allowing for improved performance. Forexample, a strict deduction that verifies that a sharpened sensorspectrum will result in a more accurate result may be provided withrespect to the Mean Value Theorem for Integrals as presented below inequation 4 and equation 5,

$\begin{matrix}{{{\int_{a}^{b}{{f(x)}{g(x)}\ {\mathbb{d}x}}} = {{f(\xi)}{\int_{a}^{b}{{g(x)}\mspace{11mu}{\mathbb{d}x}}}}},\left( {a \leq \xi \leq b} \right)} & {{EQ}.\mspace{14mu} 4}\end{matrix}$

For a given color, for example red (R), the Mean Value Theorem presentedin equation 4 may be represented as shown in equation 5.R(x,y)=∫L(λx,y)F(λ,x,y)S _(R)(λ)dλ=F(λ_(R) ,x,y)∫L(λ,x,y)S_(R)(λ)dλ  EQ. 5

In equation 5, λ_(R) is not a constant for all L(λ, x, y), but it isalways in the response range of R sensor. If S_(R)(λ) is sharp enoughand F(λ, x, y) doesn't change very fast along with wave length, thenF(λ_(R), x, y) can be seen as a constant.

FIG. 14 shows plots of the optical-to-electrical responses of the plotof FIG. 13, both before the proposed sensor response sharpening postprocessing, at 1401, and after the proposed sensor response sharpeningpost processing, at 1403. Plots 1401 and 1403 are shown relative to aplot of F(λ, x, y), at 1402, representing CMOS sensor response under thecombined effect of optical crosstalk and radial fall-off gain relativeto wavelength.

For example, the non-sharpened signal responses S_(B)(λ), S_(G)(λ) andS_(R)(λ) are shown at 1302, 1304 and 1306, respectively. The sharpenedsignal responses S_(B)(λ), S_(G)(λ) and S_(R)(λ) are shown at 1404,1406, and 1408, respectively. As further shown in FIG. 14, the rangeξ_(R) of non-sharpened S_(R)(λ) shown at 1410 corresponds to a largerange of F(λ, x, y), as shown at 1412. However, the range ξ_(R) ofsharpened S_(R)(λ) shown at 1416 corresponds to a small range of F(λ, x,y), as shown at 1414.

Based on the above description, it can now be shown that a linearcombination C_(sharp) matrix may be applied to the original sensorspectrum response, to generate sharper sensor spectrum responses ofS′_(R)(λ), S′_(G)(λ) and S′_(B)(λ), as expressed in equation 6.

$\begin{matrix}{\begin{pmatrix}{S_{R}^{\prime}(\lambda)} \\{S_{G}^{\prime}(\lambda)} \\{S_{B}^{\prime}(\lambda)}\end{pmatrix} = {C_{sharp}\begin{pmatrix}{S_{R}(\lambda)} \\{S_{G}(\lambda)} \\{S_{B}(\lambda)}\end{pmatrix}}} & {{EQ}.\mspace{14mu} 6}\end{matrix}$

Therefore, the new R, G and B values based on such sensor spectrumresponse should may be expressed as shown in equation 7, below.

$\begin{matrix}{{\begin{pmatrix}{R^{\prime}\left( {x,y} \right)} \\{G^{\prime}\left( {x,y} \right)} \\{B^{\prime}\left( {x,y} \right)}\end{pmatrix} \approx \begin{pmatrix}{\int{{L\left( {\lambda,x,y} \right)}{F\left( {\lambda,x,y} \right)}{S_{R}^{\prime}(\lambda)}{\mathbb{d}\lambda}}} \\{\int{{L\left( {\lambda,x,y} \right)}{F\left( {\lambda,x,y} \right)}{S_{G}^{\prime}(\lambda)}{\mathbb{d}\lambda}}} \\{\int{{L\left( {\lambda,x,y} \right)}{F\left( {\lambda,x,y} \right)}{S_{B}^{\prime}(\lambda)}{\mathbb{d}\lambda}}}\end{pmatrix}} = {{C_{sharp}\begin{pmatrix}{\int{{L\left( {\lambda,x,y} \right)}{F\left( {\lambda,x,y} \right)}{S_{R}(\lambda)}{\mathbb{d}\lambda}}} \\{\int{{L\left( {\lambda,x,y} \right)}{F\left( {\lambda,x,y} \right)}{S_{G}(\lambda)}{\mathbb{d}\lambda}}} \\{\int{{L\left( {\lambda,x,y} \right)}{F\left( {\lambda,x,y} \right)}{S_{B}(\lambda)}{\mathbb{d}\lambda}}}\end{pmatrix}} = {C_{sharp}\begin{pmatrix}{R\left( {x,y} \right)} \\{G\left( {x,y} \right)} \\{B\left( {x,y} \right)}\end{pmatrix}}}} & {{EQ}.\mspace{14mu} 7}\end{matrix}$

Once the new sharpened R, G, and B values are determined in accordancewith equation 7, the old radial fall-off equation presented at equationmay be rewritten as shown in equation 8, below.

$\begin{matrix}{\begin{pmatrix}{R\left( {x,y} \right)} \\{G\left( {x,y} \right)} \\{B\left( {x,y} \right)}\end{pmatrix}_{foc} = {\begin{pmatrix}{F_{R}\left( {x,y} \right)} & \; & \; \\\; & {F_{G}\left( {x,y} \right)} & \; \\\; & \; & {F_{B}\left( {x,y} \right)}\end{pmatrix}{C_{sharp}\begin{pmatrix}{R\left( {x,y} \right)} \\{G\left( {x,y} \right)} \\{B\left( {x,y} \right)}\end{pmatrix}}}} & {{EQ}.\mspace{14mu} 8}\end{matrix}$

As shown in equation 8, the approach described above may be representedas a position adaptive color correction method with the limitation thatthe color correction matrix can be split into a position adaptivediagonal matrix and a non-position adaptive matrix.

The derivation of a sensor response model is described, below, thatallows the combined effect of radial fall-off and optical crosstalk tobe effectively reduced. In the description below, input light from areal scene is depicted as L(λ, x, y), in which λ is the wavelength,(x,y) is the pixel position. Radial fall-off gain, depicted as RF(λ, x,y), is assumed to be related to wave length and pixel position, butconstant for different luminance. Individual color filters depicted asF_(R)(λ), F_(G1)(λ), F_(G2)(λ) and F_(B)(λ), may also be written as F(x,y, λ), since if x, y is known, it is known whether its value isF_(R)(λ), F_(G1)(λ), F_(G2)(λ) or F_(B)(λ). The radial optical crosstalkcoefficients may be depicted as OCT(λ, x, y, i, j), in which for “i” and“j” are integers in between −OCR and OCR, which denote a maximum opticalcrosstalk range.

Further, when crosstalk occurs, light through a current pixel's colorfilter may fall into an adjacent pixel sensor. OCT(λ, x, y, i, j) mayrepresent the percentage of light with wavelength λ that is leaked frompixel (x,y) to pixel (x+i,y+j). CMOS sensor sensitivity may be depictedas s(λ). Further, it is assumed that the color filter at the top ofpixels have three types, i.e., R, G, B, but the CMOS sensor at thebottom of pixels are assumed to be all the same. In addition, electricalcrosstalk may be denoted as ECT(i, j), in which values for “i” and “j”are integers in between −ECR and ECR, which denote a maximum electricalcrosstalk range.

Based on the above notations, the light energy passing through the colorfilter of a pixel X, pixel X may be R, G₁, G₂ or B at (x,y), may bedescribed by equation 9, below.L′(λ,x,y)=L(λ,x,y)·RF(λ,x,y)·F _(X)(λ)  EQ. 9

After consideration of optical crosstalk with respect to neighbor pixelsand optical to electrical conversion,

$\begin{matrix}{{E\left( {x,y} \right)} = {\int{{s(\lambda)}{\sum\limits_{i,{j \in {({{- {OCTR}},{OCTR}})}}}{\left\lbrack {{L^{\prime}\left( {\lambda,{x + i},{y + j}} \right)} \cdot {{OCT}\left( {\lambda,x,y,i,j} \right)}} \right\rbrack \cdot {\mathbb{d}\lambda}}}}}} & {{EQ}.\mspace{14mu} 10}\end{matrix}$

After electrical crosstalk and read out,

$\begin{matrix}{{X\left( {x,y} \right)} = {{\sum\limits_{i,{j \in {\lbrack{{- {ECTR}},{ECTR}}\rbrack}}}\left\lbrack {{E\left( {x,y} \right)} \cdot {{ECT}\left( {i,j} \right)}} \right\rbrack} + {dark\_ current}}} & {{EQ}.\mspace{14mu} 11}\end{matrix}$

Next, noise and non-linearity of optical to electrical conversion may beignored and dark current may be removed. Thus the approach may bere-written as shown in equation 12.

$\begin{matrix}{{X\left( {x,y} \right)} = {\int{\sum\limits_{i,{j \in {\lbrack{{- {CTR}},{CTR}}\rbrack}}}{\left\lbrack {{s(\lambda)}{L^{\prime}\left( {\lambda,x,y} \right)}{{CT}\left( {\lambda,x,y,i,j} \right)}} \right\rbrack \cdot {\mathbb{d}\lambda}}}}} & {{EQ}.\mspace{14mu} 12}\end{matrix}$

Where CT(λ, x, y, i, j) is the 2D-convolution of

OCT(λ, x, y, i, j) and ECT(−i,−j), and CTR=ECTR+OCTR−1.

Equation 12 may be rewritten as shown in equation 13.

$\begin{matrix}{{X\left( {x,y} \right)} = {\int{\left\{ {\sum\limits_{i,{j \in {\lbrack{{- {CTR}},{CTR}}\rbrack}}}\left\lbrack {{{L\left( {\lambda,{x + i},{y + j}} \right)} \cdot {{RF}\left( {\lambda,{x + i},{y + j}} \right)} \cdot {s(\lambda)}}{{F\left( {{x + i},{y + j},\lambda} \right)} \cdot {{CT}\left( {\lambda,x,y,i,j} \right)}}} \right\rbrack} \right\} \cdot {\mathbb{d}\lambda}}}} & {{EQ}.\mspace{14mu} 13}\end{matrix}$

Further, we can denote s(λ)·F(x, y, λ) as S(x, y, λ), or as S_(R)(λ),S_(G1)(λ), S_(G2)(λ) and S_(B)(λ) if the pixel position (x,y) is known.Also, we can denote RF(λ, x+i, y+j)·CT(λ, x, y, i, j) as RFCT(λ, x, y,i, j), so equation 13 may be rewritten as shown in equation 14.

$\begin{matrix}{{X\left( {x,y} \right)} = {\int{\left\{ {\sum\limits_{i,{j \in {\lbrack{{- {CTR}},{CTR}}\rbrack}}}{{L\left( {\lambda,{x + i},{y + j}} \right)} \cdot {S\left( {\lambda,{x + i},{y + j}} \right)} \cdot {{RFCT}\left( {\lambda,x,y,i,j} \right)}}} \right\} \cdot {\mathbb{d}\lambda}}}} & {{EQ}.\mspace{14mu} 14}\end{matrix}$

Although equation 14 provides a solution for correcting for both radialfall-off and optical crosstalk in a generated image, equation 14 wouldbe difficult to implement efficiently. A more desirable form of equation14 that would efficiently supporting Bayer-space formatted image datawould be of the format presented in equation 15, below.X _(desired)(x,y)=∫L(λ,x,y)·S(λ,x,y)·dλ  EQ. 15

Therefore, derivation of an accurate approximation of the solutionpresented in equation 14 is provided below. In the first step inderiving an accurate approximation to the solution presented in equation14 the functions Y(λ, x, y) and Y_(desire)(λ, x, y) may be defined asshown in equation 16 and equation 17, below.

$\begin{matrix}{{Y\left( {\lambda,x,y} \right)} = {\sum\limits_{i,{j \in {\lbrack{{- {CTR}},{CTR}}\rbrack}}}{{L\left( {\lambda,{x + i},{y + j}} \right)} \cdot {S\left( {\lambda,{x + i},{y + j}} \right)} \cdot {{RFCT}\left( {\lambda,x,y,i,j} \right)}}}} & {{EQ}.\mspace{14mu} 16}\end{matrix}$Y _(desired)(λ,x,y)=L(λ,x,y)·S(λ,x,y)  EQ. 17

Therefore, X(x,y) and X_(desire)(x, y) may be defined in terms offunctions Y(λ, x, y) and Y_(desire)(λ, x, y) as shown in equation 18 andequation 19,

$\begin{matrix}{{X\left( {x,y} \right)} = {\int_{blue}^{red}{{Y\left( {\lambda,x,y} \right)} \cdot \ {\mathbb{d}\lambda}}}} & {{EQ}.\mspace{14mu} 18}\end{matrix}$and,

$\begin{matrix}{{X_{desired}\left( {x,y} \right)} = {\int_{blue}^{red}{{Y_{desired}\left( {\lambda,x,y} \right)} \cdot {\mathbb{d}\lambda}}}} & {{EQ}.\mspace{14mu} 19}\end{matrix}$

Where (λ_(blue), λ_(red)) is the wave length range for the light thatcan generate a response on sensors.

Further, it is noted that if RFCT(λ, x, y, i, j) is constant for (x, y),RFCT(λ, x, y, i, j) may be re-written as RFCT(λ, i, j), and Y(λ, x, y)is a convolution between Y_(desire)(λ, x, y) and RFCT(λ, −i, −j).Therefore, Y_(desired)(λ, x, y) may be recovered from Y(λ, x, y) using ade-convolution based on a convolution kernel which has a reversed effectof RFCT(λ, −i, −j).

While an ideal de-convolution kernel can have an infinite size, inpractice a limited length de-convolution kernel can also solve theproblem very well. In the case at hand, RFCT(λ, x, y, i, j) is spatiallyvariant, but changes very slowly, and our de-convolution kernel, whichis limited in a small range which RFCT(λ, x, y, i, j) doesn't changesignificantly, so it's reasonable to believe that a spatial adaptivede-convolution kernel may be used. This means that a linear combinationof Y(λ, x, y) with spatial adaptive coefficients can approximatelyrecover Y_(desired)(λ, x, y) as shown in equation 20.

$\begin{matrix}{{Y_{desired}\left( {\lambda,x,y} \right)} \approx {\sum\limits_{i,{j \in {\lbrack{{- {IRFCTR}},{IRFCTR}}\rbrack}}}{{Y\left( {\lambda,{x + i},{y + j}} \right)} \cdot {{IRFCT}\left( {\lambda,x,y,i,j} \right)}}}} & {{EQ}.\mspace{14mu} 20}\end{matrix}$

Where IRFCT(λ, x, y, −i, −j) is an IRFCTR×IRFCTR spatial adaptivede-convolution kernel.

By integrating both sides of equation 21 from λ_(blue) to λ_(red), weobtain equation 21.

$\begin{matrix}{{X_{desired}\left( {x,y} \right)} \approx {{\quad\quad}{\int_{\lambda_{blue}}^{\lambda_{red}}{\sum\limits_{i,{j \in {\lbrack{{- {IRFCTR}},{IRFCTR}}\rbrack}}}{{Y\left( {\lambda,{x + i},{y + j}} \right)} \cdot {{IRFCT}\left( {\lambda,x,y,i,j} \right)} \cdot {\mathbb{d}\lambda}}}}}} & {{EQ}.\mspace{14mu} 21}\end{matrix}$

As described above, the “Mean Value Theorem for Integrals” may berepresented as shown in equation 22.

$\begin{matrix}{{{\int_{a}^{b}{{f(x)}{g(x)}\ {\mathbb{d}x}}} = {{f(\xi)}{\int_{a}^{b}{{g(x)}\mspace{11mu}{\mathbb{d}x}}}}},\left( {a \leq \xi \leq b} \right)} & {{EQ}.\mspace{14mu} 22}\end{matrix}$

Therefore, equation 21 may be rewritten as equation 23.

$\begin{matrix}\begin{matrix}{{{X_{desired}\left( {x,y} \right)} \approx {\sum\limits_{l,{j \in {\lbrack{{- {IRFCTR}},{IRFCTR}}\rbrack}}}{{IRFCT}\left( {\lambda_{X}, x, y, i, j} \right)}}}} \\{\int_{\lambda_{blue}}^{\lambda_{red}}{{Y\left( {{x + i},{y + j}} \right)} \cdot {\mathbb{d}\lambda}}} \\{= {\sum\limits_{l,{j \in {\lbrack{{- {IRFCTR}},{IRFCTR}}\rbrack}}}{{IRFCT}\left( {\lambda_{X},x,y,i,j} \right)}}} \\{X\left( {{x + i},{y + j}} \right)}\end{matrix} & {{EQ}.\mspace{20mu} 23}\end{matrix}$

It is noted that λ_(X) is not a constant for x, y, i, j. For example,when Y(λ, x, y) changes, λ_(X) will also change. However, is known thatλ_(blue)≦λ_(X)≦λ_(red). Therefore, if the pixel position is known, i.e.,the pixel type is known as red, green or blue, the range can be furtherlimited to the range of that red, green and blue pixel that isgenerating the response at that location.

Since IRFCT(λ, x, y, i, j) changes slowly (not across R, G1, G2, Bchannel, just inside any one of channels) along with λ, and if S_(R)(λ),S_(G1)(λ), S_(G2)(λ) and S_(B)(λ) are sharp enough, equation 23 may berewritten as equation 24.

$\begin{matrix}{{\overset{\_}{X}\left( {x,y} \right)} = {\sum\limits_{i,{j \in {\lbrack{{- {IRFCTR}},{IRFCTR}}\rbrack}}}{{{IRFCT}\left( {x,y,i,j} \right)}{X\left( {{x + i},{y + j}} \right)}}}} & {{EQ}.\mspace{14mu} 24}\end{matrix}$

Therefore, if the R, G1, G2, B sensors have sharp spectrum responses,the approximated radial fall-off and crosstalk reduction result in Bayerspace can be represented as the convolution between the captured imageand a spatial adaptive convolution kernel. The convolution kernel isspatial adaptive and is different across R, G1, G2 and B channels (thatis because λ_(blue)≦λ_(X)≦λ_(red) and the response range of S_(R)(λ),S_(G1)(λ), S_(G2)(λ) and S_(B)(λ) are different), but in each of thesefour channels, the kernel changes slowly along with the pixel position.

However, if the sensor spectrum response is not sharp enough, we maywant to try the sharpening technique described above. However, after thesharpening, the pixel value doesn't follow the equation 15 and theequation 24 will contain error.

Understanding the sensor sharpening process requires knowledge of R, G,B data for one same pixel, and now the only way to do that isinterpolation. A linear interpolation is a non-spatial-adaptiveconvolution. Combining the linear interpolation convolution, sensorsharpening matrix and CT/RF convolution kernel, still deduces the sameform of CT/RF convolution, only with a larger size of convolutionkernel.

The color temperature adaptive radial fall-off curve is the simplestform of equation 24 in which the de-convolution matrix is 1×1. Thesensor spectrum sharpening based RF/CT method in Bayer space is also oneof the forms of equation 24, in which the de-convolution matrix is 3×3.For example, equation 8 can be re-formed for red pixels in ROB formattedimago data, as shown in equation 25. Equation 8 may be similarlyre-formed for green and blue pixels.

$\begin{matrix}{{R_{foc}\left( {x,y} \right)} = {{{F_{R}\left( {x,y} \right)} \cdot C_{{{sharp},{({1,}}}{*)}} \cdot \begin{pmatrix}\frac{\begin{matrix}{{R\left( {x,y} \right){G\left( {x,{y - 1}} \right)}} + {G\left( {{x - 1},y} \right)} +} \\{{G\left( {{x + 1},y} \right)} + {G\left( {x,{y + 1}} \right)}}\end{matrix}}{4} \\\frac{\begin{matrix}{{B\left( {{x - 1},{y - 1}} \right)} + {B\left( {{x + 1},{y - 1}} \right)} +} \\{{B\left( {{x - 1},{y + 1}} \right)} + {B\left( {{x + 1},{y + 1}} \right)}}\end{matrix}}{4}\end{pmatrix}} = {\begin{pmatrix}{c_{11} \cdot {F_{R}\left( {x,y} \right)}} \\{\frac{c_{12}}{4} \cdot {F_{R}\left( {x,y} \right)}} \\{\frac{c_{12}}{4} \cdot {F_{R}\left( {x,y} \right)}} \\{\frac{c_{12}}{4} \cdot {F_{R}\left( {x,y} \right)}} \\{\frac{c_{12}}{4} \cdot {F_{R}\left( {x,y} \right)}} \\{\frac{c_{13}}{4} \cdot {F_{R}\left( {x,y} \right)}} \\{\frac{c_{13}}{4} \cdot {F_{R}\left( {x,y} \right)}} \\{\frac{c_{13}}{4} \cdot {F_{R}\left( {x,y} \right)}} \\{\frac{c_{13}}{4} \cdot {F_{R}\left( {x,y} \right)}}\end{pmatrix}^{T} \cdot \begin{pmatrix}{R\left( {x,y} \right)} \\{G\left( {x,{y - 1}} \right)} \\{G\left( {{x - 1},y} \right)} \\{G\left( {{x + 1},y} \right)} \\{G\left( {x,{y + 1}} \right)} \\{B\left( {{x - 1},{y - 1}} \right)} \\{B\left( {{x + 1},{y - 1}} \right)} \\{B\left( {{x - 1},{y + 1}} \right)} \\{B\left( {{x + 1},{y + 1}} \right)}\end{pmatrix}}}} & {{EQ}.\mspace{14mu} 25}\end{matrix}$

An approach for estimating spatial adaptive de-convolution kernelcoefficients is derive below with respect to equations 26 through 34, inwhich the following conventions are used. Neighboring pixels of a sensorin a CMOS sensor array at pixel (x,y) are numbered as pixel 1, 2, . . .N, including pixel (x,y) itself; the notation D(x, y, n),n=1˜N is usedto replace the notation IRFCT(x, y, i, j); and the neighbor pixelsvalues of pixel (x,y) in the Bayer pattern image are noted as P(x, y,n), n=1˜N.

The coefficients in the kernel change slowly along with (x,y), and theremay be big correlation among n coefficients at same pixel. Therefore,assuming that there are M parameters that are independent for eachpixel, D(x, y, n) may be expressed as shown in equation 26.

$\begin{matrix}{{\begin{pmatrix}{D\left( {x,y,1} \right)} \\{D\left( {x,y,2} \right)} \\\ldots \\{D\left( {x,y,N} \right)}\end{pmatrix}^{T} = {\begin{pmatrix}{E\left( {x,y,1} \right)} \\{E\left( {x,y,2} \right)} \\\ldots \\{E\left( {x,y,M} \right)}\end{pmatrix}^{T} \cdot W_{M \times N}}},} & {{EQ}.\mspace{14mu} 26}\end{matrix}$Where

${W_{M \times N} = \begin{pmatrix}w_{11} & w_{12} & \ldots & w_{1\; N} \\w_{21} & w_{22} & \ldots & w_{2\; N} \\\ldots & \ldots & \ldots & \ldots \\w_{M\; 1} & w_{M\; 2} & \ldots & w_{MN}\end{pmatrix}},$and |W_(M×N)|=1

By applying equation 24 to equation 26, we obtain equation 27.

$\begin{matrix}{{\overset{\_}{X}\left( {x,y} \right)} = {\begin{pmatrix}{E\left( {x,y,1} \right)} \\{E\left( {x,y,2} \right)} \\\ldots \\{E\left( {x,y,M} \right)}\end{pmatrix}^{T} \cdot W_{M \times N} \cdot \begin{pmatrix}{P\left( {x,y,1} \right)} \\{P\left( {x,y,2} \right)} \\\ldots \\{P\left( {x,y,N} \right)}\end{pmatrix}}} & {{EQ}.\mspace{14mu} 27}\end{matrix}$

Assuming that there are M images, equation 27 may be rewritten asequation 28.

$\begin{matrix}{\begin{pmatrix}{{\overset{\_}{X}}_{1}\left( {x,y} \right)} \\{{\overset{\_}{X}}_{2}\left( {x,y} \right)} \\\ldots \\{{\overset{\_}{X}}_{M}\left( {x,y} \right)}\end{pmatrix} = {\begin{pmatrix}{E\left( {x,y,1} \right)} \\{E\left( {x,y,2} \right)} \\\ldots \\{E\left( {x,y,M} \right)}\end{pmatrix}^{T} \cdot W_{M \times N} \cdot {\quad{\quad\begin{pmatrix}{P_{1}\left( {x,y,1} \right)} & {P_{2}\left( {x,y,1} \right)} & \ldots & {P_{M}\left( {x,y,1} \right)} \\{P_{1}\left( {x,y,2} \right)} & {P_{2}\left( {x,y,2} \right)} & \ldots & {P_{M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{1}\left( {x,y,N} \right)} & {P_{2}\left( {x,y,N} \right)} & \ldots & {P_{M}\left( {x,y,N} \right)}\end{pmatrix}}}}} & {{EQ}.\mspace{14mu} 28}\end{matrix}$

Equation 28 may be rewritten as equation 29.

$\begin{matrix}{\begin{pmatrix}{{\overset{\_}{X}}_{1}\left( {x,y} \right)} \\{{\overset{\_}{X}}_{2}\left( {x,y} \right)} \\\ldots \\{{\overset{\_}{X}}_{M}\left( {x,y} \right)}\end{pmatrix} \cdot {\quad{{\quad\left\lbrack {W_{M \times N} \cdot \begin{pmatrix}{P_{1}\left( {x,y,1} \right)} & {P_{2}\left( {x,y,1} \right)} & \ldots & {P_{M}\left( {x,y,1} \right)} \\{P_{1}\left( {x,y,2} \right)} & {P_{2}\left( {x,y,2} \right)} & \ldots & {P_{M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{1}\left( {x,y,N} \right)} & {P_{2}\left( {x,y,N} \right)} & \ldots & {P_{M}\left( {x,y,N} \right)}\end{pmatrix}} \right\rbrack^{- 1}\quad}{\quad\quad}{\quad{= \begin{pmatrix}{E\left( {x,y,1} \right)} \\{E\left( {x,y,2} \right)} \\\ldots \\{E\left( {x,y,M} \right)}\end{pmatrix}^{T}}}}}} & {{EQ}.\mspace{14mu} 29}\end{matrix}$If only W_(M×N) and

$\quad\begin{pmatrix}{P_{1}\left( {x,y,1} \right)} & {P_{2}\left( {x,y,1} \right)} & \ldots & {P_{M}\left( {x,y,1} \right)} \\{P_{1}\left( {x,y,2} \right)} & {P_{2}\left( {x,y,2} \right)} & \ldots & {P_{M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{1}\left( {x,y,N} \right)} & {P_{2}\left( {x,y,N} \right)} & \ldots & {P_{M}\left( {x,y,N} \right)}\end{pmatrix}$have order of M.

Similarly, for another M images,

$\begin{matrix}{\begin{pmatrix}{{\overset{\_}{X}}_{M + 1}\left( {x,y} \right)} \\{{\overset{\_}{X}}_{M + 2}\left( {x,y} \right)} \\\ldots \\{{\overset{\_}{X}}_{M + M}\left( {x,y} \right)}\end{pmatrix} \cdot \left\lbrack {{{W_{M \times N} \cdot \left. \quad\begin{pmatrix}{P_{M + 1}\left( {x,y,1} \right)} & {P_{M + 2}\left( {x,y,1} \right)} & \ldots & {P_{M + M}\left( {x,y,1} \right)} \\{P_{M + 1}\left( {x,y,2} \right)} & {P_{M + 2}\left( {x,y,2} \right)} & \ldots & {P_{M + M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{M + 1}\left( {x,y,N} \right)} & {P_{M + 2}\left( {x,y,N} \right)} & \ldots & {P_{M + M}\left( {x,y,N} \right)}\end{pmatrix} \right\rbrack^{- 1}}{\quad\quad}} = \begin{pmatrix}{E\left( {x,y,1} \right)} \\{E\left( {x,y,2} \right)} \\\ldots \\{E\left( {x,y,M} \right)}\end{pmatrix}^{T}} \right.} & {{EQ}.\mspace{14mu} 30}\end{matrix}$If only W_(M×N) and

$\quad\begin{pmatrix}{P_{M + 1}\left( {x,y,1} \right)} & {P_{M + 2}\left( {x,y,1} \right)} & \ldots & {P_{M + M}\left( {x,y,1} \right)} \\{P_{M + 1}\left( {x,y,2} \right)} & {P_{M + 2}\left( {x,y,2} \right)} & \ldots & {P_{M + M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{M + 1}\left( {x,y,N} \right)} & {P_{M + 2}\left( {x,y,N} \right)} & \ldots & {P_{M + M}\left( {x,y,N} \right)}\end{pmatrix}$have order of M.

Combining equation 29 with equation 30 results in equation 31.

$\begin{matrix}{{{\begin{pmatrix}{{\overset{\_}{X}}_{1}\left( {x,y} \right)} \\{{\overset{\_}{X}}_{2}\left( {x,y} \right)} \\\ldots \\{{\overset{\_}{X}}_{M}\left( {x,y} \right)}\end{pmatrix} \cdot}\quad}{\quad\left\lbrack {{W_{M \times N} \cdot \;\left. \quad\begin{pmatrix}{P_{1}\left( {x,y,1} \right)} & {P_{2}\left( {x,y,1} \right)} & \ldots & {P_{M}\left( {x,y,1} \right)} \\{P_{1}\left( {x,y,2} \right)} & {P_{2}\left( {x,y,2} \right)} & \ldots & {P_{M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{1}\left( {x,y,N} \right)} & {P_{2}\left( {x,y,N} \right)} & \ldots & {P_{M}\left( {x,y,N} \right)}\end{pmatrix} \right\rbrack^{- 1}} = {\left( \begin{matrix}{X_{M + 1}\left( {x,y} \right)} \\{X_{M + 2}\left( {x,y} \right)} \\\ldots \\{X_{M + M}\left( {x,y} \right)}\end{matrix} \right) \cdot \cdot \left\lbrack {{W_{M \times N} \cdot {\quad\quad}}\left. \quad\begin{pmatrix}{P_{M + 1}\left( {x,y,1} \right)} & {P_{M + 2}\left( {x,y,1} \right)} & \ldots & {P_{M + M}\left( {x,y,1} \right)} \\{P_{M + 1}\left( {x,y,2} \right)} & {P_{M + 2}\left( {x,y,2} \right)} & \ldots & {P_{M + M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{M + 1}\left( {x,y,N} \right)} & {P_{M + 2}\left( {x,y,N} \right)} & \ldots & {P_{M + M}\left( {x,y,N} \right)}\end{pmatrix} \right\rbrack^{- 1}} \right.}} \right.}} & {{EQ}.\mspace{14mu} 31}\end{matrix}$

Assuming that the images are captured under a controlled environment,and X₁(x, y), . . . X_(2M)(x, y) are all known, X _(*)(x, y) in equation31 may be replaced, resulting in equation 32.

$\begin{matrix}{{{\begin{pmatrix}{X_{1}\left( {x,y} \right)} \\{X_{2}\left( {x,y} \right)} \\\ldots \\{X_{M}\left( {x,y} \right)}\end{pmatrix} \cdot}\quad}{\quad{{\left\lbrack W_{M \times N}\quad \right. \cdot \left. \quad\begin{pmatrix}{P_{1}\left( {x,y,1} \right)} & {P_{2}\left( {x,y,1} \right)} & \ldots & {P_{M}\left( {x,y,1} \right)} \\{P_{1}\left( {x,y,2} \right)} & {P_{2}\left( {x,y,2} \right)} & \ldots & {P_{M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{1}\left( {x,y,N} \right)} & {P_{2}\left( {x,y,N} \right)} & \ldots & {P_{M}\left( {x,y,N} \right)}\end{pmatrix} \right\rbrack^{- 1}} = {\quad{\left( \begin{matrix}{X_{M + 1}\left( {x,y} \right)} \\{X_{M + 2}\left( {x,y} \right)} \\\ldots \\{X_{M + M}\left( {x,y} \right)}\end{matrix} \right){\quad{\quad{\cdot {\quad\left\lbrack {\quad{W_{M \times N} \cdot {\quad{{\quad\quad}\left. \quad\begin{pmatrix}{P_{M + 1}\left( {x,y,1} \right)} & {P_{M + 2}\left( {x,y,1} \right)} & \ldots & {P_{M + M}\left( {x,y,1} \right)} \\{P_{M + 1}\left( {x,y,2} \right)} & {P_{M + 2}\left( {x,y,2} \right)} & \ldots & {P_{M + M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{M + 1}\left( {x,y,N} \right)} & {P_{M + 2}\left( {x,y,N} \right)} & \ldots & {P_{M + M}\left( {x,y,N} \right)}\end{pmatrix} \right\rbrack^{- 1}}}}} \right.}}}}}}}}} & {{EQ}.\mspace{14mu} 32}\end{matrix}$

In the above equations, the matrix W_(M×N) is not known. In the realcase, W_(M×N) can not be directly calculated because of noise, and alsobecause the assumption of equation (22) might not be true. However, theerror may be defined as shown below in equation 33.

$\begin{matrix}{{{{{Error}\left( {x, y} \right)} = {\begin{pmatrix}{X_{1}\left( {x,y} \right)} \\{X_{2}\left( {x,y} \right)} \\\ldots \\{X_{M}\left( {x,y} \right)}\end{pmatrix} \cdot}}\quad}{\quad{{\left\lbrack {W_{M \times N} \cdot}\quad \right.{\quad\quad}\left. \quad\begin{pmatrix}{P_{1}\left( {x,y,1} \right)} & {P_{2}\left( {x,y,1} \right)} & \ldots & {P_{M}\left( {x,y,1} \right)} \\{P_{1}\left( {x,y,2} \right)} & {P_{2}\left( {x,y,2} \right)} & \ldots & {P_{M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{1}\left( {x,y,N} \right)} & {P_{2}\left( {x,y,N} \right)} & \ldots & {P_{M}\left( {x,y,N} \right)}\end{pmatrix} \right\rbrack^{- 1}} - {\quad{\left( \begin{matrix}{X_{M + 1}\left( {x,y} \right)} \\{X_{M + 2}\left( {x,y} \right)} \\\ldots \\{X_{M + M}\left( {x,y} \right)}\end{matrix} \right){{\quad{{\quad\quad} \cdot}\quad}\left\lbrack {W_{M \times N} \cdot \left. \quad\begin{pmatrix}{P_{M + 1}\left( {x,y,1} \right)} & {P_{M + 2}\left( {x,y,1} \right)} & \ldots & {P_{M + M}\left( {x,y,1} \right)} \\{P_{M + 1}\left( {x,y,2} \right)} & {P_{M + 2}\left( {x,y,2} \right)} & \ldots & {P_{M + M}\left( {x,y,2} \right)} \\\ldots & \ldots & \ldots & \ldots \\{P_{M + 1}\left( {x,y,N} \right)} & {P_{M + 2}\left( {x,y,N} \right)} & \ldots & {P_{M + M}\left( {x,y,N} \right)}\end{pmatrix} \right\rbrack^{- 1}} \right.}}}}}} & {{EQ}.\mspace{14mu} 33}\end{matrix}$

Further, the total error may be defined based on equation 33, as shownin equation 34, below.

$\begin{matrix}{{Total\_ error} = {\sum\limits_{{({x,y})} \in {image}}{{{Error}\left( {x,y} \right)}}_{2}}} & {{EQ}.\mspace{14mu} 34}\end{matrix}$

The total error may be approximated using a computer based on equation29 using the steepest falling gradient method. To increase thecomputation speed, such a calculation may be performed on a sub-sampledimage.

For purposes of explanation in the above description, numerous specificdetails are set forth in order to provide a thorough understanding ofthe described imaging device and an image crosstalk analysis device thatreduce optical crosstalk and radial fall-off in image data generatedusing an array sensor unit. It will be apparent, however, to one skilledin the art that the described imaging device and an image crosstalkanalysis device that reduce optical crosstalk and radial fall-off inimage data may be practiced without these specific details. In otherinstances, well-known structures and devices are shown in block diagramform in order to avoid obscuring the features of the described imagingdevice and an image crosstalk analysis device.

While the imaging device and an image crosstalk analysis device thatreduce optical crosstalk and radial fall-off in image data have beendescribed in conjunction with the specific embodiments thereof, it isevident that many alternatives, modifications, and variations will beapparent to those skilled in the art. Accordingly, embodiments of theimaging device and an image crosstalk analysis device that reduceoptical crosstalk and radial fall-off in image data, as set forthherein, are intended to be illustrative, not limiting. There are changesthat may be made without departing from the spirit and scope of theinvention.

What is claimed is:
 1. An imaging device, comprising: an optical sensingunit that generates image data containing a plurality of color channels;a data storage unit that stores the generated image data and a pluralityof image correction parameters; and an image processing unit thatprocesses the color channels to remove optical crosstalk and radialfall-off distortion based on the stored plurality of image correctionparameters, the image processing unit including a Bayer space adaptationunit that converts Bayer space formatted image data to R, G₁, G₂, B dataprior to a removal of optical crosstalk and radial fall-off distortionand converts the R, G₁, G₂, B data to Bayer space after the removal ofoptical crosstalk and radial fall-off distortion.
 2. The imaging deviceof claim 1, wherein the image processing unit comprises: an auto whitebalance unit that identifies color temperatures in the image data; and acolor temperature approximation unit that identifies a closest colortemperature corresponding to one or more of the plurality of imagecorrection parameters that are stored in the data storage unit.
 3. Theimaging device of claim 1, wherein the image processing unit comprises:a sensor sharpening unit that sharpens an effective sensor response foreach color channel; and a radial fall-off unit that corrects radialfall-off distortion and optical crosstalk in each color channel.
 4. Theimaging device of claim 3, wherein the plurality of image correctionparameters include: a sensor response sharpening matrix that is used bythe sensor sharpening unit to sharpen the effective sensor response foreach color channel.
 5. The imaging device of claim 4, wherein theplurality of image correction parameters include: radial fall-offcorrection parameters that are used by the radial fall-off unit tocorrect the radial fall-off distortion and optical crosstalk in eachcolor channel.
 6. The imaging device of claim 1, wherein the imageprocessing unit comprises: a de-convolution unit that corrects radialfall-off distortion and optical crosstalk in each color channel.
 7. Theimaging device of claim 6, wherein the plurality of image correctionparameters include: de-convolution coefficients that are used by thede-convolution unit to correct the radial fall-off distortion andoptical crosstalk in each color channel.
 8. The imaging device of claim1, wherein the image processing unit comprises: an electrical crosstalkcorrection unit that corrects electrical crosstalk based on one or moreof the stored image correction parameters.
 9. The imaging device ofclaim 1, wherein the image processing unit comprises: a spectralcrosstalk correction unit that corrects spectral crosstalk based on oneor more of the stored image correction parameters.
 10. The imagingdevice of claim 1, further comprising; an input/output interface thatreceives the plurality of image correction parameters from an externaldevice.
 11. An image analysis device, comprising: a sharpening matrixgeneration unit that generates a sensor sharpening matrix that sharpensan effective sensor response of a color channel in an image; a radialfall-off curve generation unit that generates a set of radial fall-offcorrection parameters that corrects optical cross-talk and radialfall-off in the color channel in the image; and a space adaptation unitthat converts image data from Bayer space format to R, G₁, G₂, B data toreduce effects of optical crosstalk and radial fall-off distortion andconverts the R, G₁, G₂, B data back to Bayer space format after thereduction.
 12. The image analysis device of claim 11, furthercomprising: a spectral crosstalk correction unit that generates colorcorrection tables to correct the spectral crosstalk in the image. 13.The image analysis device of claim 11, further comprising: an electricalcrosstalk correction unit that generates electrical crosstalk correctionparameters to correct electrical crosstalk in the image.
 14. The imageanalysis device of claim 11, further comprising: a de-convolutioncoefficient unit that generates a set of de-convolution coefficientsthat correct the optical cross-talk and radial fall-off in the colorchannel.
 15. The image analysis device of claim 11, further comprising:an input/output interface that is used to establish data communicationwith an imaging device.
 16. The image analysis device of claim 15,wherein the input/output interface is used to pass the sensor sharpeningmatrix and the radial fall-off correction parameters to the imagingdevice.
 17. The image analysis device of claim 15, further comprising: aflat-field control unit that controls generation of a plurality of colortemperature flat-field images for the imaging device.
 18. The imageanalysis device of claim 17, further comprising: a channel filtrationunit that analyses the generated color temperature flat-field images andgenerates a set of filter control parameters that filter noise and darkcurrent from the generated color temperature flat-field images.
 19. Animaging device, comprising: an optical sensing unit that generates imagedata containing a plurality of color channels; a data storage unit thatstores the generated image data and a plurality of image correctionparameters; an image processing unit that processes the color channelsto remove optical crosstalk and radial fall-off distortion based on thestored plurality of image correction parameters; and a space adaptationunit that converts image data from Bayer space format to R, G₁, G₂, Bdata to reduce effects of optical crosstalk and radial fall-offdistortion and converts the R, G₁, G₂, B data back to Bayer space formatafter the reduction.